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Linking in parametric encoding 

EPO - DG 1 
OSt 07, 2001 



The invention lelates to a linking unit according to the preamble of claim 1 . 
The linking unit serves for generating linking information indicating components of 
consecutive (typically overlapping) extended segments sp and sc which may be linked 
together in order to form a sinusoidal track, the segments sp and sc approximating 
consecutive segments of a sinusoidal audio or speech signal s. 

The invention further relates to a parametric encoder according to the 
preamble of claim 8 and a method for generating said linking information according to the 
preamble of claim 9. 



In the prior there are kno^vn two substantially different approaches for 
providing the linking information L used to establish sinusoidal tracks over consecutive 
segments. According to a first approach as described in the WO 00/79519 (PHN 017502 
EP.P) partial signals of an original audio or speech signal are reconstructed based on 
1 5 sinusoidal input data including ampUtude, frequency and phase information from a previous 
and a current segment. These reconstructed partial signals are compared with the original 
audio- or speech signal. The weighted mean-squared error signal was proposed as a criterion 
to select relevant links, i.e. to generate the linking information L. 

This first approach does not only take amplitude and frequency information 
20 into account for optimally linking consecutive segments but also considers phase information 
of the components of the previous and the current segment. However, the drawback of this 
first approach is its computational burden and the fact that the original signal is required to 
generate the linking information. 

According to a second approach known in the art the linking information is 
25 generated by only considering the amplitude and the frequency information from the 
sinusoidal code data from the current and the previous segment but not their phase 
information. Said second approach is now described by referring to Fig. 5. 

Fig. 5 shows a linking unit 500 as described in the preamble of claim 1 . It 
comprises a calculating imit 520 for generating a similarity matrix S(m,n) Ln response to 
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received sinusoidal code data Dp', Dc'. Said sinusoidal code data include information about 
the amplitudes and the frequencies of M components Xm with m = 1 ...M of the extended 
previous segment sp and of N components yn with n = 1 ...N of the extended current segment 
sc. The similarity matrix S(m,n) represents the similarity between the m'th component Xm of 
5 said extended previous segment sp and the n'th component yn of said extended current 
segment sc for m = 1 ...M and n = 1 ...N. Said similarity matrix S(m,n) is input into an 
evaluating unit 540 which evaluates said similarity matrix in order to generate said linking 
information L by selecting those pairs of components m^ the similarity of which is maximal. 

Consequently, the linking information L indicates those pairs of components 
10 of consecutive extended segments which may be linked together when restoring the audio or 
speech signal s after storage or transmission such that transitions between consecutive 
segments or components thereof are as smooth as possible. Smooth transitions lead to an 
improved quality of the restored signal. 

Hereinafter linked components continuing over consecutive segments are 
1 5 referred to as sinusoidal track even if the separate components include slight variations, e.g. 
amplitude or jfrequency variations. 

An advanced application of that second approach has been described by B. 
Edler, H. Pumhagen, and C. Ferekidis, in "ASAC-Analysis/synthesis codec for very low bit 
rates*', Preprint 4179 (F-6) 100* AES Convention, Copenhagen, 11-14 May, 1996. 
20 In that article the authors propose a combination of relative distances in 

frequency and amplitudes as an additional criterion for generating the linking ioformation. 
Expressed in other words, the linking information indicates if and which components of the 
previous and the current segment are considered to be local estimates belonging to the same 
sinusoidal crack. 

25 Advantageously according to the second approach the generation of the 

linking information is done without considering the original audio or speech signal; however, 
since generation of the linking information according to the second approach is based on 
estimated sinusoidal code data only, the generated linking information may be wrong and 
incorrect tracks may be provided. 

30 Starting fiom said second approach it is the object of the present invention to 

further develop a known linking unit, a parametric encoder and a method for generating 
linking information such that the selection of components of consecutive segments suitable 
for being linked together is improved resulting in a definition of a correct sinusoidal track. 
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That object is solved by the subject matter of claim 1 . According to the 
characterising portion of cledm 1 enlarged sinusoidal code data shall be provided comprising 
not only amplitude and frequency information but also information about the phase of at least 
some of the M components Xmand at least some of N components yn. Further, the calculation 
5 imit of a Hnk-ing unit is adapted to calciilate the similarity matrix S(m^) by additionally 
considering the phase consistency between m'th component Xm of the extended previoiis 
segment sp and the n'th compon^t yn of the extended current segment sc. 

Advantageously, the proposed linking unit does only use estimated sinusoidal 
code data including phase infoimation for generating the linking information. By additionally 
1 0 considering the phase information a more accurate determination of the similarity matrix and 
thvis, a more reliable - in comparison to the second approach known in the art - determination 
of the linking information is possible without considering the original audio or speech signal 
s. 

1 5 According to a first embodiment the calculating unit comprises a first pattem 

generating unit for generating said M complex components XmCt) of the extended previous 
segment sp and a second i>attem generating unit for generating said N complex components 
yn(t) of the extended current segment sc. The explicit calculation of these complex and time- 
dependent components is required according to the invention in order to be able to evaluate 

20 the phase consistency between each of said components of the previous and of the current 
segment 

Advantageously, the calculating module is adapted to calculate the similarity 
matrix S(m,n) as a product of a first similarity S 1 (m,n) representing the similarity in shape 
and a second similarity matrix S2(m^) representing the similarity in amplitude between the 
25 components m and n. Further, advantageous embodiments of the linking unit are subject 
matters of the dependent claims 4 to 7. 

The object of the invention is further solved by a parametric encoder 
according to claim 8 and a method for generating linking information according to claim 9. 
The advantages of the parametric encoder and of the method substantially correspond to the 
30 advantages mentioned above by referring to linking imit. 

Five figures are accompanying the description, wherein 

Fig. 1 shows a linking unit according to the invention; 

Fig. 2 shows a more detailed illustration of a calculating unit of the linking 
unit according to Fig. 1; 
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Fig. 3 illustrates the similarity of two components of two consecutive 

segments; 

Fig. 4 shows a parametric encoder according to the present invention; and 
Fig. 5 shows a linking unit known in the art. 
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Before a preferred embodiment of the invention will be described by referring 
to the figures a preliminary remark is made for providing some background information 
about the sinusoidal modelling of the signal segments in general. 

In sinusoidal modelling, the models are typically of the form (or can be 
rewritten as such) 



seg(t)-i;5RK(0} 



(0) 



15 



20 



where seg is a segment approximating or modelling a segment of a sinusoidal signal s. In 
these models the segment seg is represented by an extension as given on the right-hand sight 
of equation (1), wherein 91 denotes the real part of a complex variable and Ukare the K 
underlying sinusoidal or sinusoidal-like segment components of tihe segment seg. 

In particular, for a pure first sinusoidal model (extension), the segment's 
components are 



(1) 



with Ale (Bic and fit (real-valued) amplitude, frequency and phase, re^ectively, and j = 

25 V^. 

According to a second model the components of the segment are defined as: 



30 



(2) 



4> 
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vsliere Ak, cok and are as in the pure sinusoidal model and an additional parameter 
appears. Ok is a real parameter which captures amplitude changes within a segment 
A third, more elaborated known model based on polynomial is: 

«*(0 = I'Lb.^t" ]ckp\jX^,A (3) 

Lro=0 J L rtoO J 



with real parameters bk^ and ^ or complex amplitudes Bk^n ^^km^ * * 

Finally, according to a fourth model, the components of the segments are 



defined as: 



(0 = f,C,^t- exp{£d,^r" 1 (4) 



with real parameters 0k^ and complex parameters Ck.m- 

If two consecutive signal segments Sp and Sc (previous and current segment, 
respectively) are considered then there is typically an overlap in their support. Hereinafter Uk 
in the previous segment is denoted by Xm (m=l, ...,M) and Uk in the current segment is 

20 denoted by yn(ii=l, ...,N)- ^ order that profitable (in a coding sense) links are established, it 
seems reasonable to speak of a link between a component m fix>m Sp and a component n from 
Sc only if X(n(t) and yn(t) are similar within the overlap area. 

In the following preferred embodiments of the invention will be described by 
referring to Figs. 1 to 4. 

25 Fig. 1 shows a linking unit 100 according to the present invention . It 

comprises a calculating \mit 120 for generating a similarity matrix S(m,n) and an evaluating 
unit 140 for generating linking information L. The operation of the calculating \mit 120 
substantially corresponds to the operation of the calculating imit 520 and the operation of the 
evaluating imit 140 substantially corresponds to the operation of the evaluating unit 540 

30 known in the art and described above by referring to Fig. 5. However, there are the following 
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differences between the operation of the linking unit 100 according to the invention and the 
linking unit 500 known in the art. 

The calcxilating unit 120 does not only receive sinusoidal code data in the form 
of amplitude and frequency data of the previous and the current segment but receives 
5 enlarged sinusoidal code data further comprising information about the phase of all of the 
components Xn, of the previous segment sc and each of the N components yn of the current 
segment sc. 

Consequently, the calculating unit 120 is adapted to calculate the similarity 
matrix S(m,n) not only by considering the amplitude and frequency data but additionally by 

10 considering the phase consistency between the m'th component Xm of the extended previous 
segment sp and the n'th component ynof the extended current segment sc for m = 1 ...M and 
n = 1 ... N. The evaluating unit 140 receives and evaluates the similarity matrix S(m^) output 
from said calculating unit 120 in order to generate said linking information L by selecting 
those pairs of components (m,n) the similarity of which is maximal. 

15 Fig. 2 shows a detailed illustration of the calculating unit 120 according to the 

invention. It can be seen that the calculating unit 120 comprises a first pattern generating unit 
122 for generating said M components Xm(t) with m = 1 ...M of the extended previous 
segment sp in response to the previous segment's enlarged sinusoidal code data (Dp). 
Further, the calculating unit 120 comprises a second pattern generating xmit 124 for 

20 generating saidN components yn(t) with n = 1 ... N of the extended current segment Sc in 

response to the current segment s enlarged sinusoidal code data (Dc). Finally, the calculating 
unit 120 comprises a calculating module 126 for calculating die similarity matrix S(m^) on 
the basis of said received M components Xm(t) and of said received N components yn(t) 
according to a predefined similarity measure. Examples for the similarity measure are given 

25 below. 

The components Xrn(t) and yn(t) are explicitly generated and input to the 
calculation module 126 in order to determine the phase consistency between two components 
m and n and to use that phase consistency information for calculating the similarity matrix. 

In the following two embodiments of the invention will be described for 
30 carrying out the calculation of the similarity matrix S(m,n). Both embodiments have in 

common that the similarity matrix is preferably but not necessarily calculated by multiplying 
a first similarity matrix Si(m,n) representing the similarity in shape between the two 
components m and n with a second similarity matrix S2(m,n) representing the similarity in 
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amplitude between said components m and n. Then the similarity matrix is calculated 
according to: 



(5) 



10 



S(m,n) = 0 means that there is no link and the larger S(m^) is, the more likely 
it is that this can be exploited profitably as a link in a sinusoidal coding scheme. 

The first embodiment for calculating the similarity matrix S is based on the 
consideration of the similarity of the previous and the current segment within a complete 
overlapping area. The aim of said first embodiment is to identify components of the previous 
and the current segment which are similar. This can be done by a correlation method. Thus, 
according to the first embodiment a correlation coefficient poi^ is defined by 



20 



Y 2711 yn 



(6) 



1 5 where Xm (m = [1 ,M]) represents a set of components Xm of the previous segment Sp and yn(n 
- [UN]) represents the set of components yn of the current segment Sc. Fxirther, w(t) 
represents a window function and Exm represents the energy in the signal Xm according to: 



(7a) 



Analogously, Eyn represents the energy in the component yn according to 



(7b) 



25 Consequently, pa^ is a complex number which, for a link, should be close to 

1. Therefore, the first similarity matrix Si(m,n) is built as a (partial) similarity measure by: 



(8) 



0, 



elsewhere 



o TJX r r r\ 1 
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withO<Di< 1. 

Additionally, the equivalence in amplitude (or, more particular, in energy) can 
be taken into account by considering: 



IV„=mm^|=!^,|^k (9) 



E E 

yn xm 



gain, for a link, R should be a value close to 1 (in contrast to pta^, 1^ is real- 
valued) and as similarity measure can act S2(m,n) defined by 



[ 0, elsewhere 

withO<D2< 1. 

f the previous segment sp is represented by M components and if the cxjrrent 
1 5 segment sc is represented by N components the &st matrix Si and the second matrix S2 as 
well as the overall similarity matrix S are M x N matrices. The entries of said matrix S 
establish if there exist links and, if so, which are the most profitable ones. The most 
profitable ones are the ones the similarity values of vAnch are mayinifll. This evaluation of 
the similarity matrix S(m,n) is done in the evaluatii^ \mit 140. 
20 he second embodiment of the invention for calculating the similarity matrix S 

represents a simplification of the first embodiment. More specifically, not the whole 
overlapping region between the consecutive segment but only the ncdd point of said region is 
considered. At this point, hereinafter referred to as sample to, it is 

25 Xm(to)« ya(to) (11) 

In that second embodiment it is appreciated that in the neighbourhood of to the 
components are matched as well. This is realised if the progression (the stride) in the 
components is (nearly) the same. This is preferably evaluated by the ratio of the components 
30 of the two consecutive segments Sp and s© according to 
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xJfo+^^yAt^ (12) 

In Older to select links the first (partial) similarity matrix is now defined as: 



5 5,(i»ji) = 



1- 



0, 



-l/D„ if 



-1 



<D, 



elsewhere 



(13) 



withO<D3<l. 

Here, the amplitude similarity is involved in a relative way. This agrees with 
psycho -acoustic relevance and distance criteria. 
10 The second partial similarity matrix Si is defined as: 



1 - 



-1 



/D., if 



-1 



0, 



elsewhere 



(14) 



withO<D4< 1. 

1 5 The second embodiment for calculating the overall similarity matrix S differs 

from the first embodiment in that the components Xm and yn need only to be generated at 
specific instances, namely to and to+1 . 

Fig. 3 illustrates the operation of the linking unit of the present invention. It is 
shown that a component Xm(t) of a previous segment Sp at least partially overlaps with a 

20 component yii(t) of a consecutive current segment Sc in an overlap region OR. The calculation 
imit 120 and in particidar the calculating module 126 are adapted to analyse tilie similarity 
between these two components within the overlap region. If the two components are identical 
at least within said overlap region as shown in Fig. 3 the corresponding entry in the similarity 
matrix S(m^) would be set to one or at least close to one. The ampUtude, frequency and 

25 phase similarity would be recognised and evaluated by the evaluating unit 140 with the result 




that the linking information L generated by said evaluating unit 140 in Fig. 1 would indicate 
that these two components are local estimates belonging to the same sinusoidal track. 

Fig. 4 shows a parametric encoder 400 according to the present invention. Said 
encoder serves for encoding an audio- and/or speech signal s into a data stream ds including 
5 sinii3oidal code data and linking information. The encoder 400 comprises a segmentation unit 
410 for segmenting said signal s into at least a previous segment sp' and a consecutive 
current segment sc*. The encoder 400 further comprises a sinusoidal estimating imit 420 for 
generating said sinusoidal code data in the form of frequency, amplitude and phase data of M 
components Xm with m = 1 ... M of an extended previous segment sp approximating said 

10 segment sp' and of N components yn with n = 1 ... N of an extended current segment sc 
approximating said segment sc'. Said sinusoidal code data output from said sinusoidal 
estimating unit 420 is input to the linking unit 100 as described above by referring to Fig. 1 
for generating the linking information L. Said linking information is input into an arranging 
xmit 430 for generating the data stream by appropriately arranging or mixing, e.g. 

1 5 multiplexing the sinusoidal code data output from said sinusoidal estimating unit 420 with 
said Imking information. The arranging unit 430 is preferably embodied as multiplexer. 

For real audio signals it has been noted that taken in phase information 
improves the quality of the coded material. However, in the encoder 400 the phase 
information is used only if a continuation of a track parametric is searched. If a frequency 

20 from the data of the previous frame does not have a backward cotmection (i.e., it is not yet a 
track but may, after linking with the current frame date, become the start of a track) then the 
phase information is used but relayed on the previous linking procedures based on frequency 
and amplitude data only. The reason for this is that at the start of the track the phase is 
usually not well-defined. This means that the linking information of the previous segment sp 

25 is input to the calculating module 126 in Fig. 3 for steering purposes. 

Instead of looking at (relative) differences between complex values Xm and ym, 
also the real and imaginary parts or amplitudes and phases can be looked at and can be xised 
to construct the similarity criterion. This has the advantage that instead of the two parameters 
that control the above given similarity measure, one or more parameter per considered 

30 variable is received. Therefore, expressed in real parameters instead of complex ones, it 
typically ends up with twice as many parameters. E.g., splitting the complex signals into 
amplitudes and phases has the interesting property that it is easier that the similarity measure 
for the phases can be made fr^uency-dependent. 
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It should be noted that the above-mentioned embodiments illustrate rather than 



limit the invention, and that those skilled in the art will be able to design many alternative 
embodiments without departing from the scope of the appended claims. In the claims, any 
reference signs placed between parentheses shall not be construed as limiting the claim. The 
5 word 'comprising* docs not exclude the presence of other elements or steps than those listed 
in a claim. The invention can be implemented by means of hardware comprising several 
distinct elements, and by means of a suitably programmed computer. In a device claim 
enumerating several means, several of these means can be embodied by one and the same 
item of hardware. The mere fact that certain measures are recited in mutually different 
1 0 dependent claims does not indicate that a combination of these measures cannot be used to 
advantage. 
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CLAIMS: EPO - DG 1 
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1. Linking unit (100) for generating linking information L indicating components 

of two consecutive extended segments sp and sc which partially overlap and which may be 
linked together in order to form a sinusoidal track, the segments sp and sc approximating 
consecutive segments of a sinusoidal audio or speech signal s, the linking unit comprising: 
5 - a calculating imit (1 20) for generating a similarity matrix S(m^) in response to 

received sinusoidal code data including information about the amplitudes and the frequencies 
of M components Xm with m=l ...M of the extended previous segment sp and of N 
components yn with n=l...N of the extended current segment sc, herein the values of said 
similarity matrix represent the similarity between the m'th component Xm of said extended 

1 0 previous segment sp and the n'th component yn of said extended current segment sc for 
m=l..rM and n=l...N; and 

an evaluating unit (140) for receiving and evaluating said sunilarity matrix 
S(m^) in order to generate said linking information L by selecting those pairs of components 
(m,n) the similarity of which is maxinoal at least within the an overlapping region; 

1 5 characterised in that 

the sinusoidal code data (Dp, Dc) is enlarged by further comprising 
information about the phase of at least some of the M components Xm and at least some of 
the N components yn; 

the calculating unit (120) is adapted to calculate the similarity matrix S(m,n) 

20 by additionally evaluating the phase consistency between the m*th component Xm of the 

extended previous segment sp and the n'th component yn of the extended current segment sc. 



2. The linking imit according to claim 1, characterised in that the calculating unit 

comprises: 

25 - a &st pattern generating unit (122) for generating said M components Xni(t) 

with m=l ...M of the extended previous segment sp in response to the previous segment's 
enlarged sinusoidal code data (Dp); 
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a second pattern generating unit (124) for generating said N components yn(t) 
with n=l „-N of the extended current segment sc in response to the current segment's 
enlarged sinusoidal code data (Dc); and 

a calculation module (126) for calculating the similarity matrix S(m,n) on the 
5 basis of said received M components xm(t) and of said received N components yn(t) 
according to a predefined similarity measiire. 



3. The linking unit according to claim 2, characterised in that the calculating 

module (126) is adapted to calculate the overall similarity matrix S(m,n) according to: 
10 S(m,n)=Si(m,n)S2(m,n) 

wherein the first similarity matrix Si(m,n) represents the similarity in shape and the second 
similarity matrix S2(m,n) represents the similarity in amplitude or energy between the 
components m and n. 



15 4. The linking unit according to claim 3 , characterised in that the sumlarity 

Si(m^) is defined according to: 

'i-k, ~i|/A, z/k^-i|<D,, 



20 



5; 



with 0< Dl< 1 
and with 



wherein: 

An.n 



25 w(t) 

y*m(t) 

Exm 



0, 



elsewhere 



t 



is the similarity measure being a cross-correlation coefficient 
representing the similarity in shape between components Xni(t) 
andyn(t); 

is a window ftuiction; 

is the complex-conjugate component ym(t); 

is the energy in the signal x^ with: E^^-'Y^ w{t) x„ (f) x'^ if) ; 



Eyn 



is the energy in tiie signal y„ with: =X ^(^) >'« (0 yl (0 
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5. The linking vmit according to claim 4, characterised in that the second 

similarity 82(01^) is defined according to: 



0, 



elsewhere 



with 0 < D2 < 1 
and wherein 



J^„^=min]^,^ 



10 



6. The linking unit according to claim 3, characterised in that the calculating 

module (126) is adapted to calculate the first similarity matrix Si(m^) according to: 



15 



5i (m.«) = ^ 



1- 



0, 



-1 



/^., if 



-1 



elsewhere 



withO<D3 < 1. 



20 



7. The linking unit according to claim 6, characterised in that the calculating 

module (126) is adapted to calculate the second similarity matrix S2(m^) according to: 



1- 



-l/i>4> if 



^„(^) >'„(^o+l) 



-1 



0, 



elsewhere 



withO<D4<l. 
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8. Parametric encoder (400) for encoding an audio- and/or speech signal s into a 

datastream including sinusoidal code data and linking information L, the encoder comprising: 

a segmentation unit (410) for segmenting said signal s into at least a previous 
segment sp' and a consecutive partially overlapping current segment sc'; 
5 " a sinusoidal estimating unit (420) for generating said sinusoidal code data in 

the form of frequency and ampUtude data of M components Xm with m==l...M of an ejrtended 
previous segment sp approximating said segment sp' and of N components yn with n=l ...N of 
an extended current segment sc approximating said segment sc'; 

a calculating unit (120) for generating a similarity matrix S(m,n) in response to 
1 0 said received smusoidal code data wherem the values of said similarity matrix represent the 
similarity between the m'th component Xm of said extended previous segment sp and the n'th 
component yn of said consecutive extended current segment sc for m=l...M and n=l...N; 

an evaluating unit (140) for receiving and evaluating said similarity matrix 
S(m,n) in order to generate said linking information L indicating those pairs of components 
1 5 m,n the similarity of which is maximal; 

an arranging unit (430) for generating the datastream representing the original 
audio- or speech signal by appropriately arranging said amplitude, frequency and linking 
information; 
characterised in that 

20 - the sinusoidal code data estimating unit (420) is adapted to fiither generate 

mformation about the phase of at least some of the M components Xm and of at least some of 
the N components yn; and 

the calculation unit (1 20) is adapted to calculate the similarity matrix S(m,n) 
by additionally considering the phase consistency between the m'th component x^ of the 

25 extended previous segment sp and the n'th component yn of the extended current segment sc. 

9* Method for generating linking information L indicating components of 

consecutive partially overiapping extended segments sp and sc which may be linked together 
in order to form a sinusoidal track, the segments sp and sc approximating consecutive 
30 segments of a sinusoidal audio-/or speech signal s, the method comprising the steps of: 

providing sinusoidal code data including information about tibie amplitudes and 
the frequencies of M components Xm with m=L..M of the extended previous segment sp and 
of N components yn with n=l...N of the extended current segment sc; 
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calculating the similarity matrix S(m^) according to a predetennined 

similarity measure wherein the similarity matrix represents the similarity between the m'th 

component Xm of said extended previous segment sp and the n^th component yn of said 

extended current segment sc for m=l...M and n=l...N; and 

evaluating said similarity matrix S(m^) in order to generate said linking 

information L by selecting those pairs of components m and n the similarity of which is 

maximal; 

characterised in that 

the step of providing the sinusoidal code data further includes the provision of 
information about the phase of at least some of the M components Xm and of at least some of 
the N components yn; and 

the similarity matrix S(m,n) is calculated by additionally considering the phase 
consistency between the n'th component yo of the extended previous segment sp and the m'th 
component Xm of the extended current segment sc. 
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The invention relates to a linking unit 100, a parametric encoder 400 and a 
method for generating linking information L indicating components of consecutive extended 
segments sp and sc which may be Unked together in order to form a sinusoidal track. The 
segments sp and sc approximate consecutive segments of a sinusoidal audio or speech signal 
5 s. The linking unit comprises a calculating unit 120 for generating a similarity matrix S(m,n) 
in response to received sinusoidal code data and an evaluating unit 140 for receiving and 
evaluating said similarity matrix S in order to generate said linking information by selecting 
those pairs of components m,n tiie similarity of which is maxinud. According to the invention 
the calculating unit 120 is adapted to calculate the similarity matrix S by additionally 
1 0 considering information about the phase consistency between the components of the 

extended previous segment sp and the extended current segment sc. In that way the selection 
of components suitable for being linked together is improved resulting in the definition of 
correct tracks. 
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